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CHAPTER  1 


INTRODUCTION 

This  research  explored  two  methods  for  identifying  aircraft 
equipment  types  where  applied  research  might  best  be  concentrated  to 
improve  equipment  reliability  on  future  aircraft.  The  research  deter¬ 
mined  if  different  identification  algorithms  emphasized  different  types 
of  equipments.  Chapter  I  provides  background  to  the  problem,  summarizes 
the  two  approaches,  compares  the  results  obtained  from  each,  identifies 
explanations  other  than  the  differences  in  algorithms  which  could  ac¬ 
count  for  the  observed  difference  between  the  two  sets  of  results,  and 
states  the  research  questions. 


BACKGROUND 

In  1973,  Johnson  and  Reel,  In  their  study  of  the  Impact  of  main¬ 
tainability  and  reliability  on  support  cost,  pointed  out  that  more  than 
two  thirds  of  the  national  defense  budget  for  the  past  six  years  has 
been  required  to  support  the  existing  military  inventory  of  equipment 
(10:1).  They  went  on  to  point  out,  "if  this  trend  is  allowed  to  con¬ 
tinue,  support  costs  could  equal,  or  exceed  the  amount  of  dollars  pre¬ 
sently  allocated  for  national  defense  (10:1)." 

The  National  Academy  of  Sciences  found  in  1975  ".  .  .  that  a  very 
Important  factor  in  the  force  of  military  aircraft  is  the  low  reliabil¬ 
ity  ..  .  (12:1)."  The  Academy  went  on  to  suggest  that  poor  reliability 
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contributed  to  the  cost  of  ownership  In  three  ways:  (1)  the  direct  cost 
of  maintaining  the  unreliable  equipment;  (2)  the  Indirect  cost  of  down¬ 
time;  and  (3)  the  indirect  cost  of  equipment  failure  during  a  mission 
(12:1). 

In  a  1977  study  by  the  Air  Force  Aeronautical  Systems  Division, 
the  total  cost  penalty  of  support,  nonavailability  (downtime),  and 
aborted  missions  was  estimated  to  be  $19  billion  for  the  remaining  lives 
of  18  style  types  and  series  of  Air  Force  aircraft.  Of  this  total:  (1) 
$8.1  billion  was  for  support  cost;  (2)  $10.3  billion  was  for  nonavaila¬ 
bility;  and  (3)  $0.5  billion  was  for  aborted  missions  (7:6). 

Johnson  and  Reel  suggested  that  "data  derived  from  specific 
equipment/  system  programs  showed  that  designed  efforts  to  increase 
reliability  ...  can  significantly  reduce  equipment/ system  life  cycle 
costs  (10.1)."  They  went  on  to  point  out  that 

Emphasis  in  this  area  [reliability]  may  result  in  higher 
costs  during  RDT&E  [Research,  Development,  Test,  and  Evaluation] 
but  any  increased  front  end  costs  are  offset  many  times  by  long¬ 
term  savings  in  man-hours  and  material  when  properly  applied  [10:23]. 

They  suggested,  however,  that  "...  it  is  not  appropriate  or  economical 
to  apply  the  same  level  of  effort  on  all  equipment  (10:23)." 

In  a  similar  vein,  a  1977  Productivity,  Reliability,  Availabil¬ 
ity,  and  Maintainability  (PRAM)  Program  Office  study  suggested  that 
".  .  .  it  is  not  entirely  obvious,  with  our  essentially  fixed  Research 
and  Development  budget,  what  types  of  equipment  should  research  be 
concentrated  on  (13:1)."  The  study  also  stated. 

It  is,  we  believe,  the  responsibility  of  the  logistic  support 
community  to  identify  the  areas  where  the  potential  exists  for  a 
high  return  on  investment  for  basic  research  to  improve  reliability 
[13:1]. 
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The  overall  problem  then,  was  to  identify  those  types  of  equip¬ 
ment  where  it  would  be  best  for  the  Air  Force  to  invest  its  limited 
reliability  improvement  funds  In  during  RDT&E. 

APPROACHES  TO  THE  PROBLEM 

A  1975  Rand  Report  by  Dr.  Fiorello  and  Mrs.  Dey  pointed  out  that 
an  "important  step  in  determining  how  to  improve  the  management  of  costs 
of  ownership  is  to  analyze  historical  data  on  operational  weapon  systems 
(6:iii).“  The  1977  Aeronautical  Systems  Division  study  mentioned  above, 
also  suggested  that  "this  [sic]  data  [on  operational  aircraft  by  equip¬ 
ment  type]  in  much  greater  detail  could  indicate  areas  where  research 
.  .  .  might  best  be  concentrated  (7:7.)"  Increased  Reliability  of 

Operational  Systems  (IROS)  and  Logistics  Investment  Screening  Techniques 
(LIST),  are  two  current  approaches  to  identifying  reliability  improve¬ 
ment  candidates  based  on  data  obtained  from  current  aircraft. 

Highlights  of  these  approaches  are  discussed  below.  A  more 
detailed  discussion  of  IROS  and  LIST  are  contained  in  appendices  A  and  B 
respectively. 

Increased  Reliability  of  Operational  Systems  (IROS-LSC) 

An  early  attempt  to  use  detailed  data  from  operational  aircraft 
to  objectively  guide  the  reliability  Investments  was  initiated  in  1970, 
and  was  called  IROS  (Increased  Reliability  of  Operational  Systems). 
While  IROS  has  several  different  data  outputs,  the  most  widely  used 
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product  was  the  Logistics  Support  Cost*  (LSC)  Ranking  described  in  AFLCM 
66-18,  Programming  and  Technical  Processes  (16:17-1  to  17-17).  The 
IROS-LSC  ranking  was  generated  by  listing  equipment  by  the  estimated, 
annual  logistics  support  cost  of  every  work  unit  coded  equipment  on 
operational  Air  Force  aircraft  (over  120,000  coded  equipments).  Rou¬ 
tinely  available  data  were  used  to  estimate  the  annual  logistics  support 
cost.  These  data  consisted  of  field  reports  on  the  maintenance  man¬ 
hours  expended  on  each  equipment  that  was  repaired  or  replaced,  and  the 
number  of  equipments  sent  to  central  depots  for  repair.  These  data  were 
combined  with  standard  field  labor  rates,  standard  shipping  costs  per 
pound,  and  standard  depot  repair  costs  for  each  of  the  equipments  re¬ 
paired  at  the  central  depots  (16:17-1  to  17-17).  Lists  of  top  100  and 
top  500  ranked  equipments  were  generated  by  IROS-LSC. 

In  his  1978  masters  thesis  at  Ohio  State  University,  Spray  con¬ 
cisely  stated. 

Very  simply,  the  idea  of  a  high  burner  list  (such  as  the  IROS- 
LSC  ranking)  Is  that  with  so  many  equipment  items  and  so  few  en¬ 
gineers  and  investment  dollars,  why  not  concentrate  our  efforts  on 
those  Items  creating  the  most  havoc  .  .  .?  Why  waste  time  or  money 
on  Items  that  are  causing  little  or  no  difficulties  [15:10]? 

He  went  on  to  say  that  "the  primary  advantage  of  the  high  burner  ap¬ 
proach  was  that  It  was  almost  totally  objective  (15:10)." 

The  high  burner  approach,  as  exemplified  by  the  IROS-LSC  "top  500 
equipments,"  was  not  without  its  critics. 


*The  term  Logistics  Support  Costs,  as  used  in  this  paper,  refers  to  the 
maintenance  costs  charged  to  or  prorated  to  a  specific  equipment.  It  includes 
costs  for  field  labor  and  materials,  second  destination  transportation  to  and 
from  central  repair  depots,  and  the  cost  of  labor  and  materials  for  repair  or 
replacement  of  items  at  the  central  depots. 
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Meltzler,  In  a  1976  study  clearly  Identified  a  potential  weakness. 


.  .  .  [A]  serious  problem  Is  that  equipment  performing  some 

aircraft  functions  Is  Inherently  more  expensive  than  that  for  other 
functions.  Thus,  we  might  not  be  surprised  If,  for  example,  the 
most  reliable  and  lowest  support  cost  Inertial  navigation  system  In 
the  Inventory  costs  more  to  support  than  the  worst  UHF  radio.  If 
this  was  actually  the  case,  should  we  consider  replacing  or  mod¬ 
ifying  the  lowest  cost,  most  reliable  Inertial  navigation  system  In 
the  Inventory  before  we  consider  acting  on  the  worst  UHF  radio? 
[11:1-2]. 


As  pointed  out  by  Or.  Womer,  now  at  Clemson  University, 

.  .  .  high  burner  lists  assume,  by  default,  equal  Investment 
for  all  Items.  This  wouldn't  be  so  bad  except  all  the  high  cost, 
high  complexity  devices  are  concentrated  at  the  top  of  the  list, 
thus  blatently  violating  this  assumption,  since  with  other  things 
being  equal,  one  would  expect  the  Investment  to  be  greatest  for  the 
most  complex,  expensive  Items  [15:11]. 

Logistics  Inventory  Screening  Technique  (LIST-TD) 

Recognizing  these  potential  difficulties  with  the  IROS-LSC  high 
burner  list,  the  PRAM  Program  Office  in  1976,  with  the  assistance  of  the 
Air  Force  Institute  of  Technology,  Initiated  the  development  of  the 
Logistics  Investment  Screening  Technique-Total  Degradation  (LIST-TD). 

LIST-TD  was  very  similar  to  IROS-LSC,  except  for: 

1.  Quantification  of  the  value  of  aircraft  downtime  and  air¬ 
craft  missions  aborted  (both  measured  in  dollars),  and 

2.  Calculation  of  the  estimated  investment  required  to  Improve 
the  reliability  of  specific  equipment. 

Since  Its  Inception,  LIST-TD  has  been  used  by  about  two  dozen 
organizations  (8:3).  These  have  Included  the  Avionics  and  Materials 
Laboratories,  Industries  such  as  General  Dynamics,  Lockheed,  Parker- 
Hannafln  (valves  and  actuators).  Grimes  (aircraft  lights),  the  A-10 
System  Program  Office,  etc.  (8:3).  Systems  based  on  LIST-TD  have  been 
Implemented  at  General  Dynamics  and  Fairchild. 
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A  COMPARISON  OF  THE  RESULTS  FROM  CURRENT  APPROACHES 

IROS-LSC  AND  LIST-TD  were  two  major  competing  approaches  to 
screening  equipment  areas  for  improved  reliability.  The  laboratories 
and  others  responsible  for  allocating  research  funds  should  be  concerned 
about  the  difference  between  the  types  of  equipments  that  were  high  on 
the  IROS-LSC  and  LIST-TD  rankings.  If  the  top  IROS-LSC  and  LIST-TD 
rankings  both  suggested  the  same  types  of  equipment  for  investment,  then 
no  problem  existed  from  the  viewpoint  of  reliability  Investment  In 
RDT&E.  On  the  other  hand,  if  differences  in  top  IROS  and  LIST  rankings 
of  equipments  were  not  the  result  of  incidental  differences  such  as 
different  data  bases,  time  period,  etc.,  then  questions  need  to  be 
raised  about  the  direction  and  thrust  of  the  Air  Force's  program  to 
Improve  the  reliability  of  future  aircraft. 

Typical  of  the  results  obtained  from  the  IROS/LSC  approach  was  a 
1975  McDonnell -Douglas  Aircraft  Corp.  study.  This  study  was  based  on  an 
IROS-LSC  ranking  of  all  aircraft  equipment  types  except  engines.  McDon¬ 
nell  -Douglas  found  an  ",  .  .  almost  total  absence  of  components  [equip¬ 
ments]  In  the  other  nonavionics  subsystems  (4:56)."  This  study  suggest¬ 
ed  that  reliability  Improvement  emphasis  should  be  placed  on  avionics. 

Several  studies  Indicated  that  there  were  considerable  differ¬ 
ences  between  the  IROS-LSC  and  LIST-TD  top  ranked  equipments.  The 
Aeronautical  Systems  Division  (ASD)  study,  previously  cited,  found  that 
with  the  LIST-TD  top  ranked  Items  (l.e.  high  benefl t- to-cost  ratio)  were 
low  to  medium  value  ($100  -  $2000)  items  which  caused  unusually  high 
degradations.  (7:21). 


To  gain  a  different  perspective  on  the  ASD  findings,  in  1977 
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Houston  analyzed  the  top  ranked  LIST-TD  equipments  by  stock  class  .  Her 
findings  are  summarized  In  Table  1  (9:2-5). 

Except  for  Bombing/Fire  Control,  her  results,  in  sharp  contrast 
to  the  McDonnell -Douglas  findings,  were  notlcably  devoid  of  avionics. 

In  another  study  by  the  Air  Force  Institute  of  Technology,  Baker 
and  Hollingsworth  re-examined  the  top  ranked  LIST  equipments  and  con¬ 
firmed  Houston's  finding  that  the  stock  class  of  valves  and  actuators 
contained  the  greatest  number  of  equipments  (1:15). 


RANK 

STOCK 

CLASS 

NAME 

NO.  OF 
EQUIPMENTS 

1 

1650 

Hydraulic  Valves  ... 

41 

2 

1280 

8ombing/Fire  Control 

22 

3 

6680 

Flow  Instruments 

19 

4 

1660 

Air  Conditioning  ... 

17 

5 

6610 

Flight  Instruments 

16 

TABLE  1:  TOP  FIVE  LIST  STOCK  CLASSES 
ALTERNATIVE  EXPLANATION  FOR  THE  OBSERVED  DIFFERENCES 

Differences  in  top  ranked  equipment  types  between  IROS-LSC  and 
LIST-TD  have  been  observed  in  the  studies  referenced  in  the  preced¬ 
ing  section. 

^Federal  Stock  Class  (FSC)  Is  defined  to  be  the  first  four  digits  of 
the  stock  number  used  In  the  supply  system  to  identify  a  specific  equip¬ 
ment.  The  FSC  Is  a  grouping  of  equipment  with  similar  characteristics  or 
functions  (e.g.,  FSC  1280,  Bombing  and  Fire  Control  Items). 


However,  it  would  have  been  risky  to  conclude  that  these  differences 
were  due  to  the  different  algorithms  used  in  IROS-LSC  and  LIST-TD. 
Table  2  summarizes  incidental  (i.e.,  unintended)  differences  between  the 
LIST-TO  and  IROS-LSC  study  methodologies.  These  could  have  produced 
effects  which  were  confounded  with  the  Intended  effects  of  the  different 
algorithms.  Five  of  the  more  obvious  incidental  differences  are  dis¬ 
cussed  below. 

One  obvious,  but  essentially  incidental  difference  was  that 
IROS-LSC  results  have  always  been  of  equipments  classified  by  work  unit 
code  (usually  the  two  digit  code)  while  the  LIST-TD  results  most  studied 
have  been  of  equipments  classified  by  federal  stock  class.  There  was  no 
general  correspondence  between  these  two  equipment  classification  schemes. 


IROS-LSC 

LIST-TD 

Data 

1974 

1976 

Aircraft 

Types 

41  aircraft  and 
hel icopters 

31  fixed  wing 
aircraft 

Equl pment 

Classification 

Work  Unit  Code 

Federal  Stock 
Class 

Equipment  Types 

All  Work  Unit 

Coded  Equipment 

20%  of  the 

Work  Unit  Coded 
Equipment 

Benef  i  t 

Weights 

Annual 

Remaining 

Life  of  Aircraft 

LSC  Data 

Quarterly 

Annual 

Number  of  Top 

Ranked  Equipments 

500 

200 

TABLE  2: 

SEVEN  DIFFERENCES  WITH 
POSSIBLE  CONFOUNDING  EFFECTS 
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Another  obvious  but  incidental  difference  was  that  the  IROS-LSC 
results  were  based  on  all  the  work  unit  coded  equipments  on  Air  Force 
aircraft,  while  LIST-TD  was  based  on  about  20%  of  the  work  unit  coded 
items  due  to  the  necessity  of  using  an  incomplete  work  unit  code  to 
national  stock  number  cross-reference  (8:1).  The  method  by  which  the 
cross-reference  was  formed  guarantees  that  this  20%  was  not  a  random 
sample,  but  was  in  fact  biased.  In  fact.  Spray  found  that  the  equip¬ 
ments  cross-referenced  (20%  of  the  total)  accounted  for  almost  50%  of 
the  support,  availability,  and  abort  costs  (13:120).  If  there  were  no 
bias  20%  of  the  equipments  should,  of  course,  capture  20%  of  the  cost. 

Another  incidental  difference  was  that  IROS-LSC  drew  on  data  from 
41  different  aircraft  types,  while  LIST-TD  drew  on  only  31  types  (7:5). 

Not  only  was  the  sample  of  aircraft  different,  but  in  each  case, 
the  time  period  covered  by  the  data  was  different. 

Finally,  IROS-LSC  used  annual .weights  for  its  logistics  support 
cost  estimates,  while  LIST-TD  estimates  of  total  degradation  were  weight¬ 
ed  based  on  the  estimated  remaining  lives  of  each  aircraft  type  (11:2). 
This  difference  would  not  affect  the  relative  ranking  of  equipment  if 
only  a  single  type  of  aircraft  were  considered.  However,  when  multiple 
types  of  aircraft  were  analyzed  IROS-LSC  Implicitly  weighted  all  air¬ 
craft  types  equally,  while  LIST-TD  weighted  them  by  the  expected  re¬ 
maining  life.  In  the  LIST  studies,  the  equipment  on  F-106,  for  in¬ 
stance,  received  only  a  small  fraction  of  the  weight  of  the  equipment  on 
an  F-15  or  even  a  B-52,  because  the  projected  remaining  life  of  the 
F-106  Is  much  less  than  that  of  either  the  F-15  or  the  B-52.  The  use  of 
different  weights  constituted  an  Incidental  and  even  an  erroneous  dif¬ 
ference  when  comparing  the  results  of  the  two  ranking  methods.  In  this 
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case  either  the  expected  life,  a  standard  life  (e.g.,  10  year  life),  or 
annual  values  should  be  used  In  both  methodologies  (l.e.,  equal  weights). 

There  were  a  number  of  alternative  explanations  that  could  have 
accounted  for  the  observed  differences.  It  should  not  have  been  con¬ 
cluded  that  a  true  or  Intended  (i.e.,  nonincidental)  difference  existed 
between  the  IROS-LSC  and  LIST-TD  algorithms  until  an  unbiased  comparison 
under  properly  controlled  conditions  was  made. 

The  objective  of  this  thesis  was  to  perform  a  controlled  com¬ 
parison  test  between  IROS-LSC  and  LIST-TD  in  order  to  determine  if  the 
different  ranking  algorithms  caused  the  observed  differences  in  the  top 
ranked  equipments. 


RESEARCH  QUESTIONS 

Previous  studies  showed  that  the  types  of  top  ranked  IROS-LSC  and 
LIST-TD  equipments  were  different.  It  was  not  known,  however,  whether 
these  differences  were  due  to  incidental  differences  between  the  studies 
listed  in  Table  2,  or  to  the  fundamental  differences  between  the  IROS- 
LSC  and  LIST-TD  algorithms.  If  all  such  incidental  differences  were 
eliminated  would  there  have  been  a  significant  and  practical  difference 
between  top  ranked  IROS-LSC  and  LIST-TD  equipment  types?  If  so,  this 
difference  could  only  have  been  attributed  to  the  fundamental  differ¬ 
ences  in  the  ranking  algorithms. 

Primary  Research  Question 

The  primary  research  question  was: 

Oo  differences  in  the  IROS-LSC  and  LIST-TD  ranking  algorithms 
cause  significant  differences  in  the  resulting  fop  ranked  IROS-LSC 
and  LIST-TD  equipments? 
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Secondary  Research  Questions 

There  are  other  questions  of  interest  if  the  differences  in 
top  ranked  equipments  were  not  explained  by  incidental  causes. 

1.  Given  an  overall,  nonincidental  difference  between  the  top 
ranked  IROS-LSC  and  LIST-TD  equipments,  which  specific  equipment 
types  are  significantly  different? 

The  thrust  of  this  question  was  to  identify  which  equipment  types 
were  emphasized  by  IROS-LSC,  and  which  types  were  emphasized  by  LIST-TD. 
While  it  might  be  informative  to  know  that  the  types  of  top  ranked 
IROS-LSC  and  LIST-TD  equipments  are  truly  different,  it  was  of  practical 
importance  to  know  which  types  of  equipment  were  significantly  empha¬ 
sized  by  IROS-LSC  as  compared  to  LIST-TD,  and  vice  versa. 

2.  Given  an  overall,  nonincidental  difference,  was  this  due, 
at  least  in  part,  to  the  estimated  investment  feature  of  LIST-TD? 

If  the  types  of  equipments  emphasized  were  significantly  different 
when  the  return  on  investment  alogrithm  was  used,  this  might  suggest 
that  more  consideration  should  be  given  to  the  question  of  estimated 
investments  in  reliability  Improvement. 


3.  Given  an  overall  significant  difference  and  a  significant 
estimated  Investment  effect,  does  the  inclusion  by  LIST-TD  of  bene¬ 
fits  due  to  reduced  downtime  and  aborted  missions,  make  a  signifi¬ 
cant  difference? 


Specifically,  this  question  pertained  to  whether  or  not  there  was 
a  significant  difference  between  top  ranked  LIST-TD  equipments  where 
only  support  cost  benefits  were  considered,  as  opposed  to  the  case  where 
only  downtime  and  abort  benefits  were  considered.  If  a  significant 
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difference  existed,  this  would  be  of  interest  because  it  could  imply 
that  the  types  of  equipments  one  would  invest  in  to  reduce  support  cost 
are  different  from  those  that  one  would  invest  in  to  reduce  downtime  and 
aborts. 


CHAPTER  SUMMARY 

There  was  considerable  concern  about  the  high  equipment  support 
costs,  the  excessive  downtime,  and  the  high  mission  abort  rates  of 
modem  weapon  systems,  such  as  Air  Force  aircraft. 

It  has  been  suggested  that  poor  equipment  reliability  was  an 
important  root  cause  of  this  situation.  Research  to  improve  equipment 
reliability  would  have  high  payoff  on  future  aircraft  if  the  research 
were  concentrated  on  the  appropriate  types  of  equipment.  Central  to  the 
use  of  research  to  improved  equipment  reliability,  was  a  method  of 
Identifying  types  of  equipment  with  the  highest  potential  payoff. 

Data  on  fielded  aircraft  were  used  to  identify  equipment  types 
for  reliability  investment.  One  approach  to  doing  this  was  the  IROS-LSC 
(Increased  Reliability  of  Operational  Systems  -  Logistic  Support  Cost) 
which  simply  ranks  equipments  by  their  quarterly  logistics  support 
costs.  Another  somewhat  similar  approach,  LIST-TD  (Logistics  Investment 
Screening  Technique  -Total  Degradation),  also  ranks  equipments,  but  in 
so  doing  LIST-TD  considers  the  investment  required  to  make  equipments 
more  reliable  and  also  combines  the  benefits  of  reduced  support  cost 
with  those  of  decreased  maintenance  and  supply  downtimes  and  reduced 
preflight  and  Inflight  aborts. 

The  quarterly  lists  of  top  500  equipments  generated  by  the  IR0S- 
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LSC  approach  emphasized  expensive  equipment  types  such  as  jet  engines 
and  radar  sets  while  the  LIST-TD  approach,  as  analyzed  by  Houston 
(9:2-5),  emphasized  more  “mundane"  equipment  types  such  as  valves  and 
actuators.  The  conditions  under  which  these  results  were  obtained  were 
not  entirely  comparable.  The  IROS-LSC  results  were  reported  by  work 
unit  code  while  LIST-TO  results  were  reported  by  federal  stock  class. 
The  specific  aircraft  and  equipments  in  the  data  bases  were  different  as 
were  the  time  periods  of  the  data.  Finally,  logistic  support  costs  were 
calculated  in  IROS-LSC  using  annual  cost  values,  while  LIST-TD's  calcu¬ 
lations  were  based  on  the  remaining  aircraft  life. 

Previous  studies  showed  that  the  types  of  top  ranked  IROS-LSC  and 
LIST-TD  equipments  were  different:  It  was  not  known,  however,  whether 
these  differences  were  due  to  incidental  differences  among  the  studies, 
or  to  the  fundamental  differences  between  the  IROS-LSC  and  LIST-TD 
algorithms.  If  all  such  incidental  differences  were  eliminated,  would 
there  still  be  a  significant  and  practical  difference  between  top  ranked 
IROS-LSC  and  LIST-TD  equipment  types?  This  difference  could  only  be 
attributed  to  the  fundamental  differences  in  the  ranking  algorithms. 

The  primary  research  question  was  "can  the  observed  differences 
in  top  ranked  IROS-LSC  and  LIST-TD  equipment  types  be  explained  by  the 
incidental  differences  In  data  samples,  use  of  different  remaining 
lives,  etc.,  or  was  there  an  intended,  significant  difference  due  to  the 
Inclusion  In  LIST-TD  of  estimated  investments  and/or  the  additional 
benefits  of  reduced  downtime  and  aborts?" 

Given  the  background  leading  to  the  problem  and  the  establishment 
of  the  research  questions,  the  next  task  was  to  develop  a  research 
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approach  to  answering  the  research  questions.  This  is  accomplished  in 
Chapter  2.  The  results  of  the  research  are  presented  in  Chapter  3, 
while  conclusions  and  recommendations  are  stated  in  Chapter  4,  the  final 
chapter. 


CHAPTER  2 


RESEARCH  METHODOLOGY 

In  Chapter  1  the  following  primary  research  question  was  devel¬ 
oped: 

Do  differences  in  the  IROS-LSC  and  LIST-TD  ranking  algorithms 
cause  significant  differences  in  the  top  ranked  IROS-LSC  and  LIST-TD 
equipments? 

Depending  on  the  answer  to  this  question,  secondary  research  questions 
are  of  interest.  In  this  chapter,  the  experimental  design  used  to 
answer  the  primary  question  is  developed.  Next  the  experimental  con¬ 
trols  used  to  control  the  confounding  factors  noted  in  the  first  chapter 
are  discussed.  After  a  description  of  the  sample  used  in  the  research, 
the  critical  research  question  is  stated  as  a  statistically  testable 
hypothesis.  In  the  next  section  the  methodology  used  to  answer  the 
supplemental  research  questions  Is  presented. 

The  final  section  of  this  chapter  explains  the  exact  computa¬ 
tional  procedures  used  In  testing  this  hyopthesis. 

EXPERIMENTAL  DESIGN 

A  valid  comparison  between  IROS-LSC  and  LIST-TD  must  eliminate, 
as  much  as  possible,  all  Incidental  influences  while  preserving  the 
nonincidental,  intended  differences  In  the  ranking  algorithms.  If  the 
Incidental  Influences  were  not  controlled  in  the  experimental  design, 
they  might  produce  effects  confounded  with  the  effects  of  the  intended 
experimental  treatment  (14:5).  In  the  classical  experiment,  assessment 


of  the  effect  of  a  treatment  would  be  made  by  running  the  controlled 
experiment  with  and  without  the  treatment.  The  outcomes  would  be  ob¬ 
served  and  recorded.  The  two  groups  of  outcomes  then  would  be  compared. 
If  a  statistically  significant  difference  existed,  it  would  be  presumed 
due  to  the  treatment. 

While  the  analogy  of  the  classical  experiment  to  this  research 
situation  was  not  exact,  it  was  of  explanatory  value.  The  incidental 
effects  were  considered  extraneous  variables  requiring  control.  The  "no 
treatment”  case  considered  the  logistics  support  cost  (IROS-LSC)  ranking 
algorithm.  The  "treatment”  case  was  the  list  ranking  algorithm  which 
had  an  Return  On  Investment  feature  and  included,  in  addition  to  LSC, 
downtime  and  abort  costs.  The  distributions  by  stock  class  of  the  top 
500  ranked  equipments  in  each  case  were  analogous  to  "outcomes."  A 
statistical  comparison  between  the  two  distributions  of  top  500  ranked 
equipments  by  stock  class  was  employed  to  discern  significant  effects  in 
the  different  algorithms.  This  methodology  was  schematically  depicted 
in  Figure  1. 

This  experiment  was  capable  of  replication  by  using  data  from 
other  time  periods.  Also,  this  experiment  could  be  repeated  under 
widely  differing  circumstances  to  check  for  wider  generalizations.  For 
example,  while  this  "experiment"  used  only  Air  Force  aircraft,  there  was 
no  reason  why  it  could  not  be  repeated  on  Navy,  RCAF  aircraft,  etc. 

While  the  design  of  the  critical  experiment  was  straight  forward. 
Its  success  was  entirely  dependent  on  identifying  the  major  incidental 
influences  and  controlling  them,  so  that  any  observed  differences  could 
only  be  due  to  the  nonincidental,  intended  differences  between  the 
IRO-LSC  and  LIST-TD  algorithms. 
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EXPERIMENTAL  CONTROLS 


Seven  Incidental  influences  were  identified  in  the  first  chapter 
that  could  have  caused  the  previously  observed  differences  between 
IROS-LSC  and  LIST-TO.  These  incidental  differences  are  discussed  below, 
together  with  the  approach  taken  to  control  each. 

1.  Different  Time  Periods  -  The  data  used  in  previous  compari¬ 
sons  came  from  different  time  periods.  For  example  the  data  used  by 
Houston  in  her  LIST-TD  study  (9:1-5)  were  compiled  using  data  available 
two  years  older  than  the  McDonnell  aircraft  IROS-LSC  data.  Specific 
changes  such  as  missions,  major  modifications,  or  aircraft  maturation 
might  produce  effects  confounded  with  the  intended  effects  of  the  dif¬ 
ferent  algorithms.  To  control  for  this,  data  from  the  same  time  period 
were  used,  eliminating  this  as  a  potential  cause  of  any  observed  differ¬ 
ence  between  the  IROS-LSC  and  LIST-TD  algorithms. 

2.  Different  Aircraft  -  In  previous  comparisons,  the  data 
utilized  were  drawn  from  different  types  of  aircraft.  The  McDonnell 
aircraft  study,  for  instance,  considered  fighter  type  aircraft,  while 
the  Houston  study  contained  a  mix  of  fighter,  bomber,  and  cargo  air¬ 
craft.  If  particular  type  of  equipment  and/or  the  failure  rates  were 
significantly  different  among  aircraft  types,  these  could  produce  ef¬ 
fects  confounded  with  the  Intended  effects  of  the  different  algorithms. 
The  control  used  in  the  present  research  was  to  draw  IROS-LSC  and  LIST- 
TD  data  from  an  identical  set  of  aircraft. 

3.  Time  Duration  -  IROS-LSC  results  have  always  been  based  on 
a  single  quarter  (three  months)  of  data.  LIST-TD,  on  the  other  hand, 
has  always  employed  four  quarters  of  data.  The  difference  could  have 
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had  an  unintended,  Incidental  effect  on  previous  comparisons  between 
IROS-LSC  and  LIST-TD.  This  was  controlled  by  using  four  quarters  of 
data  for  both  IROS-LSC  and  LIST-TD. 

4.  Equipment  Classification  -  IROS-LSC  results  have  always 
been  expressed  by  work  unit  code  (one  or  two  digit).  LIST-TD  results 
have  usually  been  expressed  by  stock  class,  as  in  the  Houston  and  Baker 
and  Hollingsworth  studies.  As  there  was  not  a  direct  correspondence 
between  the  work  unit  code  and  stock  class,  the  previously  observed 
differences  could  have  been  due  to  the  use  of  different  schemes  for 
classifying  the  results.  By  using  either  the  work  unit  code  or  the 
stock  class  classification  scheme,  but  not  both,  control  can  be  a- 
chleved.  The  exclusive  use  of  stock  classes  was  arbitrarily  chosen  for 
control  in  this  study. 

6.  Different  Equipments  -  Even  for  the  same  time  period,  the 
same  aircraft,  and  the  same  time  duration,  the  actual  equipments  in  the 
data  base  for  IROS-LSC  and  LIST-TD  would  be  considerably  different.  The 
equipments  in  the  LIST-TD  data  base  are  actually  a  subset  of  the  equip¬ 
ments  in  the  IROS-LSC  data  base.  LIST-TD  contains  about  20%  of  the 
equipments  in  the  IROS-LSC  data  base.  LIST-TD  must  use  a  cross  refer¬ 
ence  between  work  unit  codes  and  master  stock  numbers  in  order  to  obtain 
a  unit  price  for  the  equipment.  The  cross  reference  used  by  LIST-TD  was 
developed  by  the  Logistics  Management  Institute,  and  it  does  not  contain 
equipments  that  fail  infrequently.  This  was  controlled  for  by  using 
only  equipments  with  a  valid  cross  reference  between  work  unit  code  and 
master  stock  number. 

6.  Assumed  Lives  -  IROS-LSC  ranking  algorithms  did  not  con¬ 
sider  the  aircraft  lives  over  which  the  benefits  were  amortized.  LIST- 


19 


TD,  on  the  other  hand,  explicitly  made  an  assumption  about  remaining 
aircraft  lives.  In  the  Houston  and  Aeronautical  System  Division  studies 
cited  earlier,  the  official,  planned  remaining  lives  were  used.  Other 
things  being  equal,  equipment  on  aircraft  with  long  remaining  lives  will 
be  ranked  higher  than  equipment  on  aircraft  with  short  remaining  lives. 
This  difference  in  weighting  could  have  accounted  for  the  observed  dif¬ 
ference  between  IROS-LSC  and  LIST-TD  top  ranked  equipment  types.  It  was 
controlled  for  by  assuming  a  ten  year  life  for  all  aircraft  in  both 
IROS-LSC  and  LIST-TD. 

7.  Number  of  Top  Ranked  Equipments  -  In  previous  studies,  the 
number  of  equipments  defined  as  "top  ranked"  varied  from  one  study  to 
the  next.  A  larger  number  would,  of  course,  allow  more  lower  ranked 
equipments  to  be  included,  possibly  of  different  types.  This  could  have 
accounted  for  the  observed  difference  between  IROS-LSC  and  LIST-TD. 
This  was  controlled  for  by  ranking  exactly  500  equipments. 

With  the  experimental  design  and  controls  specified,  the  actual 
sample  used  in  the  experiment  is  defined  in  the  next  section. 

SAMPLE 

The  universe  of  the  critical  experiment  was  all  equipment  on  all 
currently  fielded  US  Air  Force  aircraft.  The  population  was  all  work 
unit  coded  equipments  on  the  31  fielded  aircraft  contained  in  the  LIST- 
TD  data  base,  a  subset  of  those  contained  in  the  IROS-LSC  data  base. 

The  sample  was  all  equipments  on  the  31  fielded  US  Air  Force  air¬ 
craft  with  a  valid  cross  reference  reported  on  during  the  period,  1 
October  1976  to  31  September  1977.  There  were  exactly  24,685  valid 
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cross  referenced  equipments.  Each  one  represented  a  specific  work  unit 
coded  equipment  type  on  a  specific  aircraft  type. 

The  primary  research  question  developed  in  Chapter  1  lacks  the 
operational  definitions  required  to  be  statistically  tested.  In  the 
next  section  the  primary  research  question  is  restated  as  a  precise, 
statistically  testable  hypothesis. 

PRIMARY  HYPOTHESIS 


Having  described  the  experimental  design,  experimental  controls, 
and  sample  the  primary  research  question  can  now  be  stated  as  a  precise 
and  statistically  testable  hypothesis: 

Hfl:  The  difference  In  the  federal  stock  class  distribution 
of  the  500  top  ranked  IROS-LSC  equipments  as  compared  to  the  500  top 
ranked  LIST-TD  equipments  was  not  significantly  different. 

Hj :  The  difference  was  significantly  different. 

Significance  was  determined  by  the  application  of  the  Chi-Square 
statistic  to  a  contingency  table  formed  by  the  stock  class  frequencies 
of  top  ranked  IROS-LSC  and  LIST-TD  equipments.  The  significance  level 
was  .01(2:139). 

If  the  null  hypothesis  was  not  rejected,  then  the  other  research 
questions  are  moot.  The  research  conclusion  would  have  been  that  no 
significant  difference  existed  between  the  top  ranked  IROS-LSC  and 
LIST-TD  equipment  types.  From  this.  It  would  follow  that  the  previously 
observed  differences  were  entirely  incidental. 

Rejection  of  the  null  hypothesis  would  Infer  that  a  true  signifi¬ 
cant  difference  existed  between  top  ranked  IROS-LSC  and  LIST-TD  equip¬ 
ment  types  for  the  sample  examined.  From  this,  it  would  follow  that  at 
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least  some  of  the  previously  observed  differences  between  IROS-LSC  and 
LIST-TD  results  were  not  Incidental.  If  the  null  hypothesis  was  re¬ 
jected,  the  secondary  research  questions  would  have  to  be  addressed  to 
determine  more  specifically,  the  source  of  the  difference. 

SECONDARY  RESEARCH  QUESTIONS 

Having  defined  the  experimental  design  and  the  primary  hypo¬ 
thesis,  the  methodology  used  in  answering  the  secondary  research  ques¬ 
tions  follows  directly.  Discussion  is  limited  to  those  key  features 
differing  from  the  primary  research  question.  In  all  cases  it  is  as¬ 
sumed  that  the  null  hypothesis  in  the  primary  hypothesis  was  rejected. 

Significantly  Different  Equipment  Types 

The  first  secondary  question  was,  "Which  particular  federal  stock 
classes,  taken  one  at  a  time,  show  a  significant  difference  between  top 
ranked  IROS-LSC  and  LIST-TD  equipment  types?" 

This  question  was  answered  by  assuming  for  each  stock  class  that 
no  difference  exists  (Hq),  and  then  examining  the  observed  difference. 
If  the  difference  between  the  assumed  and  observed  frequencies  was 
significant  then  reject  the  null  hypothesis.  The  Chi-Square  test  was 
used  to  compare  frequencies.  On  these  individual  tests  the  level  of 
significance  was  set  at  .01  (2:139). 

Re turn -On- Investment 

The  next  secondary  question  was,  "Does  the  return-on-1 nvestment 
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feature  (i.e.,  the  estimated  investment  feature)  of  LIST-TD  cause  a 
significance  between  top  ranked  IROS-LSC  and  LIST-TD  equipment  types?'* 

This  question  was  answered  by  performing  an  experiment  exactly 
like  the  critical  experiment  in  every  way  except  one.  The  single  dif¬ 
ference  was  to  leave  out  the  added  benefits  of  reduced  downtime  and 
aborts  in  LIST-TD  (i.e.  the  only  benefit  used  was  the  logistics  support 
cost).  Significance  was  determined  by  application  of  the  Chi-Square 
Test  to  the  Contingency  Table.  The  significance  level  was  .01  (12:139). 

Failure  to  reject  the  null  hypothesis  would  have  led  to  the 
conclusion  that  the  difference  In  the  critical  hypothesis  (e.g.,  between 
IROS-LSC  and  LIST-TD)  was  not  due  to  the  return-on-investment  feature  of 
LIST.  To  reject  the  null  hypothesis  would  imply  that  the  difference  in 
the  critical  hypothesis  was  due  at  least  in  part  to  the  return-on-invest¬ 
ment  feature. 

Added  LIST  Benefits 

Did  the  LIST-TD  feature  of  Including  downtime  and  abort  costs 
make  a  significant  difference?  If  equipments  that  have  high  support 
costs  always  had  high  downtime  and  abort  costs,  then  there  would  not 
have  been  a  significant  difference  between  equipment  types  ranked  by 
support  cost  and  those  ranked  by  downtime  plus  abort  costs. 

The  significance  of  adding  the  downtime  and  abort  costs  was 
determined  by  repeating  the  return  on  Investment  experiment  with  one 
difference.  In  this  experiment  the  comparison  was  between  LIST-LSC 
(I.e.,  the  only  benefit  used  was  LSC)  and  LIST-AB,  where  AB  stands  for 
"added  benefits."  LIST-AB  was  defined  as  the  sum  of  downtime  cost  and 
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abort  cost  divided  by  the  estimated  investment  for  each  equipment. 

Since  on  any  specific  given  equipment,  the  estimated  investment, 
I,  mbs  the  same  regardless  of  whether  one  considers  the  support  cost  of 
the  added  benefits  (e.g.  downtime  and  abort  costs),  differences  result¬ 
ing  from  the  above  experiment  were  attributed  to  the  Inclusion  of  the 
different  benefits.  Significance  was  determined  by  application  of  the 
Chi-Square  Test  to  the  Contingency  Table.  The  significane  level  was  .01 
(12:139). 

A  failure  to  reject  Hq  would  have  implied  that  there  was  no 
significant  difference  among  top  ranked  equipment  types  if  one  con¬ 
sidered  logistics  support  costs  vis-a-vis  downtime  and  abort  costs. 
Rejection  would  have  implied  that  the  difference  between  IROS-LSC  and 
LIST-TD  was  due  at  least  in  part  to  the  different  types  of  costs,  namely 
logistics  support  costs  versus  downtime  and  abort  costs.  Considerable 
emphasis  has  been  placed  by  top  Air  Force  managers  on  improving  aircraft 
availability.  If  the  equipment  types  emphasized  to  reduce  support  costs 
were  considerably  different  than  the  equipment  types  emphasized  to 
reduce  downtime  and  aborts,  then  working  on  the  former  might  not  do  much 
for  the  latter. 


COMPUTATIONAL  PROCEDURES 

For  the  critical  experiment,  the  data  tape  containing  the  24,685 
equipments  in  the  sample  was  processed  on  a  CDC  6600  computer  to  obtain 
the  500  top  ranked  IROS-LSC  and  LIST-TD  equipments  as  specified  pre¬ 
viously.  These  two  groups  of  500  equipments  were  each  sorted  by  stock 
class  and  the  two  groups  were  printed.  The  number  of  equipments  In  each 
Federal  Stock  Class  were  then  counted  and  the  frequencies  in  each  stock 
class  recorded. 
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STOCK  IROS  LIST 

CLASS 

1650  A  E 

1280  B  F 

6680  C  G 

1660  D  H 

TABLE  3: 

SAMPLE  CONTINGENCY  TABLE 

The  results  were  formed  Into  a  contingency  table  similar  to  the 
one  shown  in  Table  3.  In  Table  3  the  upper  case  letters  in  each  cell 
represent  the  observed  cell  frequencies. 

Siegel  gave  some  special  considerations  in  applying  the  Chi- 
Square  (X  )  test  to  contingency  tables  with  more  than  one  degree  of 
freedom. 

2 

The  X  test  may  be  used  if  fewer  than  20  percent  of  the  cells 
have  an  expected  frequency  of  less  than  5  and  If  no  cell  has  an 
expected  frequency  of  less  than  1.  If  these  requirements  are  not 
met  by  the  data  In  the  form  In  which  they  were  originally  collected, 
the  researcher  must  combine  adjacent  categories  in  order  to  increase 
the  expected  frequencies  in  the  various  cells.  Only  after  he  has 
combined  categories  to  meet  the  above  requirements  may  he  meaning¬ 
fully  apply  the  tr  test.  [14:110] 

To  Insure  the  Chi-Square  test  was  meaningfully  applied,  all  cells 
with  an  expected  frequency  of  less  than  two  were  combined  into  a  single 
miscellaneous  class  (stock  class  0000).  Additionally  when  the  number  of 
cells  with  expected  frequencies  less  than  five  and  greater  than  one, 
exceeded  20X  of  the  total  cells,  stock  classes  with  the  fewest  observa¬ 
tions  where  combined  with  stock  class  0000  until  these  conditions  were 
met. 

^Underlining  added  by  the  authors. 
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The  Chi-Square  value  for  each  stock  class  was  then  calculated  as 


X2 

*i 
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(f2-feL) 


2 


where 

2 

Xj-  was  the  Chi-Square  value  for  stock  class  i 

fj  was  the  IROS-LSC  observed  frequency  for  stock  class  i 
fg  was  the  LIST-TD  observed  frequency  for  stock  class  i . 
f  j  was  the  expected  frequency  for  the  IROS-LSC  for  stock  class  i. 
fg,  was  the  expected  frequency  for  the  LIST-TD  for  stock  class  i. 


F  .j  was  calculated  by  multiplying  the  IROS/LSC  marginal  proba¬ 
bility  by  the  total  observed  frequency  for  stock  class  i.  Because  the 
same  number  of  observations  was  used  in  both  the  IROS-LSC  and  LIST-TD, 
the  marginal  probability  for  both  was  equal  to  .5.  FfiL  was  calculated 
by  multiplying  the  LIST-TD  marginal  probability  by  the  total  observed 
frequency  for  stock  class  i . 

The  total  X  value  was  simply  the  sum  of  the  individual  values 
across  all  stock  classes  (treating  the  combined  "000"  stock  class  as  a 
single  class). 

*2-  £«? 

i*i 

where 

X  was  the  total  Chi-Square  value 

n  was  the  number  of  unique  stock  classes  (treating  the  combined 

miscellaneous  as  a  single  class) 
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2 

The  total  X  value  was  then  compared  with  the  critical  value  from 
the  statistical  tables  (Fisher  and  Yates)  for  .01  significance  and 
[(2-1)  x  (n-1)]  degrees  of  freedom.  The  null  hypothesis  was  not  re¬ 
jected  if  the  total  Chi-Square  value  was  less  than  the  table  critical 
value.  The  null  hypothesis  was  rejected  if  the  total  value  was  greater 
than  the  critical  value  (14:106). 

In  the  event  the  null  hypothesis  was  rejected,  each  stock  class 
can  then  be  tested  individually  for  a  significant  difference  by  noting 
whether  its  Chi-Square  value  exceeds  that  of  the  critical  value  from  the 
table  for  .01  level  of  significance  and  1  degree  of  freedom.  This  value 
was  6.64. 

If  the  null  hypothesis  was  rejected  on  the  primary  experiment, 
then  this  entire  process  would  be  repeated  for  the  secondary  experiments 
with  the  treatments  being  changed  as  previously  described. 

CHAPTER  SUMMARY 

As  an  aid  to  understanding  the  research  approach,  the  IROS-LSC 
and  LIST-TD  processes  were  summarized.  In  an  uncontrolled  comparison 
between  IROS-LSC  and  LIST-TD,  there  were  seven  major  incidental  dif¬ 
ferences  that  could  affect  the  results. 

A  design  for  an  ’'experiment,''  similar  in  many  respects  to  the 
classical  controlled  experiment,  was  developed.  It  was  shown  how  this 
design  controlled  for  the  major  incidental  effects. 

The  universe,  population,  sample,  and  other  terms  were  then  given 
operational  definitions.  With  these  in  hand,  the  primary  and  secondary 
questions  were  restated  as  mathematical  hypotheses.  Finally,  the  exact 
mathematical  procedures  used  in  testing  the  hypothesis  were  given. 


CHAPTER  3 


RESULTS  AND  CONCLUSIONS 
PRIMARY  HYPOTHESIS 


It  will  be  recalled,  from  Chapter  2,  that  the  primary  hypothesis 


was: 


Hfl:  The  difference  in  the  federal  stock  class  distribution 

of  the  500  top  tanked  IROS-LSC  equipments  as  compared  to  the  500  top 
ranked  LIST-TD  equipments  was  not  significantly  different. 

H^:  The  difference  was  significantly  different. 

This  hypothesis  was  tested  with  the  Chi-Square  statistic  at  the 
.01  significance  level  as  defined  in  Chapter  2.  The  details  of  the 
Chi-Square  calculations  are  given  in  Appendix  C. 

The  calculated  Chi-Square  value  was  342.3.  For  a  .01%  level  of 
significance  and  39  degrees  of  freedom,  the  critical  Chi-Square  value 
was  62.4.  As  342.3  was  greater  than  62.4,  the  null  hypothesis,  HQ,  was 
rejected,  and  the  alternative  hypothesis,  Hj,  accepted. 

The  consequence  of  such  an  outcome  was  given  in  Chapter  2  as: 

Rejection  of  the  null  hypothesis  would  infer  that  a  true  signi¬ 
ficant  difference  exists  between  top  ranked  IROS-LSC  and  LIST-TD  e- 
quipment  types  for  the  sample  examined. 

From  this,  it  would  follow  that  at  least  some  of  the  previously 
observed  differences  between  IROS-LSC  and  LIST-TD  results  were  not 
incidental.  If  the  null  hypothesis  was  rejected,  the  secondary 
research  questions  would  have  to  be  addressed  to  determine  more 
specifically,  the  source  of  the  difference. 

The  conclusion  was  limited,  in  that  it  only  applied  to  the  ex¬ 
perimental  sample.  More  generalized  conclusions,  and  the  risks  Inherent 
in  such  generalizations  are  considered  in  the  next  chapter. 
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SECONDARY  RESEARCH  QUESTIONS 

As  the  null  hypothesis  in  the  primary  hypothesis  was  rejected, 
the  secondary  research  questions  were  addressed. 

Significantly  Oifferent  Equipment  Types 

As  stated  In  Chapter  2,  the  first  secondary  question  was: 

Which  particular  stock  classes,  taken  one  at  a  time,  show  a 
significant  difference  between  top  ranked  IROS-LSC  and  LIST-TD  e- 
quipment  types? 

The  stock  classes  that  were  significantly  different  are  shown  in 
Table  4  below.  Those  stock  classes  emphasized  by  IROS-LSC  (i.e.  those 
stock  classes  with  the  greatest  frequency  of  occurence)  were  shown  in 
the  left  column,  while  those  emphasized  by  LIST-TD  were  shown  in  the 
right  column.  It  can  readily  be  seen  that  IROS-LSC  emphasized  jet 
engines  and  avionics  while  LIST-TD  emphasized  valves,  actuators,  fuel 
gauges,  air  conditioning  and  other  miscellaneous  or  "mundane"  items. 
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IROS-LSC  EMPHASIS  (.01% 


LIST-TD  EMPHASIS  (.01% 


STOCK 

STOCK 

CLASS 

EQUIPMENT  TYPE 

CLASS 

EOUIPMENT  TYPE 

1270 

Fire  Control 

1620 

Landing  Gear 

1430 

Missile  Remote  Control 

1650 

Hydraulic  Valves 
and  Actuators 

1610 

Props 

1660 

Air  Conditioning 

2840 

Jet  Engines 

1680 

Mi  sc.  Accessories 

5821 

Radios 

2620 

Tires 

5841 

Radar 

6340 

Alarms 

5865 

Electronic  Counter 

Measures 

6620 

Engine  Instrm. 

6605 

Navigation 

6680 

Fuel  Gauges 

6615 

Autopilot 

TABLE  4:  SIGNIFICANTLY  DIFFERENT  EQUIPMENT  TYPES 


Return-On- Investment 


The  second  supplemental  question  was: 

Does  the  return-on-investment  feature  (i.e.,  the  estimated 
investment  feature)  of  LIST-TD  cause  a  difference  between  top  ranked 
IROS-LSC  and  LIST-TD  equipment  classes? 

This  was  tested  by  comparing  IROS-LSC  with  LIST-LSC,  where  the 
later  was  identical  to  IROS-LSC  except  the  logistic  support  costs  for 
each  equipment  were  divided  by  the  estimated  investment,  i.e.,  LSC/I. 

The  resulting  Chi-Square  value  (see  Appendix  C  for  details)  was 
297.7.  As  297.7  was  greater  than  the  critical  Chi-Square  value  of  62.4 
(for  a  .01%  level  of  significance),  the  null  hypothesis  that  there  was 
no  significant  difference  was  rejected.  This  led  to  the  conclusion,  as 
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stated  In  Chapter  2,  that 


The  difference  in  the  critical  hypothesis  was  due  at  least  in 
part  to  the  return-on-investment  feature  [of  LIST-TD]. 

Having  rejected  the  null  hypothesis,  the  individual  stock  classes 
were  examined  for  a  significant  difference.  Those  stock  classes  with  a 
significant  difference  (.012  level  of  significance)  were  shown  in  Table 
5.  As  before,  when  IROS-LSC  had  the  greatest  frequency,  the  stock  class 
was  listed  in  the  left  column,  while  LIST-LSC  results  were  shown  in  the 
right  column.  For  the  sample  examined,  IROS-LSC  again  primarily  empha¬ 
sized  jet  engines  and  avionics,  although  pumps  and  engine  assessories 
were  now  included.  LIST-LSC  emphasized  an  assortment  of  "mundane" 
items. 
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IROS-LSC  EMPHASIS  (.01%)  LIST-TD  EMPHASIS  (.01%) 

STOCK  STOCK 


CLASS 

EQUIPMENT  TYPE 

CLASS 

EQUIPMENT  TYPE 

1270 

Fire  Control 

1620 

Landing  Gear 

1430 

Missile  Remote  Control 

1630 

Wheels  and  Brakes 

2840 

Jet  Engines 

1650 

Hydraulic  Valves 
and  Actuators 

2995 

Engine  Assessories 

1680 

Mi  sc.  Assessories 

4320 

Pumps 

2620 

Tires 

5821 

Radios 

6220 

Lights 

5841 

Radar 

5885 

Electronic  Counter 

Measures 

6605 

Navigation 

' 

6615 

Autopilot 

TABLE  5:  RETURN-ON- INVESTMENT  RESULTS 


Added  LIST  Benefits 

The  last  supplemental  research  question  given  in  Capter  2  was: 

Did  the  LIST-TD  feature  of  adding  in  downtime  and  abort  costs 
make  a  significant  difference? 


The  calculated  Chi-Square  value  (see  Appendix  C  for  details)  was 
149.7.  As  this  was  greater  than  the  critical  Chi-Square  value  of  64.2 
(for  a  .01X  level  of  significance),  the  null  hypothesis  that  there  was 
no  difference  was  rejected. 

As  was  stated  In  Chapter  2: 

Rejection  of  the  null  hypothesis  would  have  Implied  that  the 
difference  between  IROS-LSC  and  LIST-TD  [at  least  for  this  sample] 
was  due  at  least  In  part  to  the  different  types  of  costs,  namely 
logistics  support  costs  versus  downtime  and  abort  costs. 
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Because  the  null  hypothesis  was  rejected,  the  individual  stock 
classes  were  examined  for  significant  differences.  The  results  are 


shown  in  Table  6. 

LIST-LSC  EMPHASIS  (99%)  LIST-AB  EMPHASIS  (99%) 


SC 

EOUIPMENT  TYPE 

SC 

EOUIPMENT  TYPE 

1640 

Batteries 

6610 

Flight  Instruments 

2620 

Tires 

4920 

Maint  Special 
Equipment 

1630 

Wheels  and  Brakes 

4810 

Valves 

2915 

Engine  Fuel  System 

1660 

Air  Conditioning 

TABLE  6:  ADDED  BENEFITS  RESULTS 
CHAPTER  SUMMARY 

For  the  sample  examined  it  was  found-  that: 

1.  There  was  a  significant  difference  between  top  ranked  IROS- 
LSC  and  LIST-TD  equipment  types. 

2.  That  this  difference  was  due  to  both  the  inclusion  in  the 
LIST-TD  algorithm  of  an  estimated  investment  and  the  added  benefits 
(reduced  cost)  of  downtime  and  aborts. 

3.  That  the  types  of  equipments  emphasized  by  IROS-LSC  were  jet 
engines  and  avionics,  while  those  emphasized  by  LIST-TD  were  mundane 
items  such  as  hydraulic  valves  and  actuators,  fuel  gauges,  etc. 

The  final  chapter  considers  the  generalizations  of  these  findings 
beyond  the  sample  examined. 
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CHAPTER  4 


RISKS,  CONCLUSIONS,  AND  RECOMMENDATIONS 

In  Chapter  2,  specific  experiments  were  devised  to  address  the 
research  questions.  Chapter  3  presented  results  for  a  particular  sample 
of  24,685  equipments.  While  these  specific  results  may  be  generally 
valid  when  applied  to  other  periods,  other  aircraft,  etc.,  there  were  a 
number  of  risks  inherent  in  such  generalizations.  The  acceptability  of 
these  risks  depends,  in  the  end,  on  the  judgment  of  each  user  of  the 
results  of  this  research. 

The  risks  to  generalization  are  given  below,  followed  by  some 
conclusions  (keeping  the  risks  in  mind),  and,  finally,  some  recommend¬ 
ations. 


RISKS  TO  EXTERNAL  VALIDITY 

Four  potential  sources  of  risk  In  generalizing  the  experimental 
results  will  be  discussed. 

(1)  If  Instead  of  comparing  the  stock  class  frequencies  of  the 
top  500  In  each  case,  the  top  100,  top  1000,  etc.,  had  been  compared, 
the  result  might  have  been  different. 

(2)  The  use  of  only  WUC/MSN  cross  referenced  equipments  means 
that  the  "IROS-LSC"  results  of  the  "experiment"  will  be  different,  to 
some  extent,  from  the  true  IROS-LSC  results  based  on  all  equipments. 
There  were  grounds  for  believing  that  this  could  Introduce  a  consider¬ 
able  bias  favoring  equipments  with  frequent  failures  and  maintenance 
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actions.  While  such  a  bias  would  certainly  seriously  affect  the  outcome 
of  an  experiment  intended  to  gauge  the  precise  change  in  an  interval  or 
ratio  level  parameter,  it  was  difficult  to  conceive  of  any  systematic 
bias  in  this  case  that  could  mask  a  difference  in  the  ordinal  level 
results  or  falsely  create  a  difference  where  one  did  not  truly  exist. 

(3)  The  one  year  of  data  used  might  not  be  representative,  and 
the  use  of  data  from  some  other  year  could  change  the  conclusions. 

(4)  The  input  data  gathered  and  consolidated  by  various  auto¬ 
mated  systems  contained  errors.  It  was  possible  that  some  systematic 
error  could  have  interacted  with  the  treatment  in  such  a  way  as  to 
falsely  create  an  apparent  difference  when  in  fact  there  was  none,  and 
vice  versa.  For  example,  errors  in  the  stock  list  prices  (e.g.  such 
that  they  were  extremely  low)  would  cause  the  estimated  investment  to  be 
understated.  Since  investment  was  a  divisor  in  the  treatment  case,  the 
rank  would  be  falsely  high.  As  there  would  not  be  a  similar  effect  in 
the  untreated  case,  this  would  be  an  interaction  effect  with  the  treat¬ 
ment  and  a  source  of  invalid  difference. 

CONCLUSIONS 

As  discussed  above,  there  were  a  number  of  risks  to  making  any 
generalizations.  The  large  sample  of  over  24,000  equipments,  the  use  of 
controls  in  the  quasl-experimental  design,  and  the  use  of  the  .01  level 
of  significance  should,  however,  have  reduced  such  risks.  Until  the 
experiment  is  independently  replicated  and  rerun  under  other  conditions 
using  other  aircraft  from  other  time  periods,  no  generalization  will  be 
completely  without  risk. 


With  full  knowledge  of  these  risks,  the  authors  concluded  that  it 
was  highly  probable  that  top  ranked  IROS-LSC  and  LIST-TD  equipment  types 
would  be  found  to  be  significantly  different  for  all  reasonably  sized 
samples  of  Air  Force  aircraft  equipments.  This  significant  difference 
would  be  due  to  both  the  Investment  and  the  added  benefit  features  of 
LIST-TD.  These  were  the  principle  conclusions  of  this  research.  It  was 
the  authors'  Intent  to  show  that  for  at  least  one  high  burner  algorithm 
(IROS-LSC)  and  one  estimated  return-on-investment  algorithm  (LIST-TD), 
that  the  results  obtained  were  significantly  different  and  that  this 
difference  was  of  sufficient  magnitude  to  be  of  concern  to  Air  Force 
decision  makers. 


RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

As  with  most  research,  one  possibility  for  further  research  Is 
replication  using  data  from  different  time  periods,  different  aircraft, 
etc.  Such  replications  would  either  strengthen  or  weaken  the  conclu¬ 
sions  reached  In  this  research.  While  not  discouraging  such  research, 
it  may  be  more  fruitful  to  consider  some  of  the  larger  questions  that 
remain. 

It  would  be  most  useful  to  know  whether  the  significant  differ¬ 
ences  established  by  this  research  were  due  to  peculiarities  In  the 
algorithms  used  by  LIST-TD  to:  (1)  estimate  the  added  benefits  of 
downtime  and  aborts;  and  (2)  estimate  the  Investment  required  to  improve 
equipment  reliability.  A  number  of  alternatives  to  the  highly  simpli¬ 
fied  LIST-TD  algorithms  have  been  suggested,  and  it  would  be  most  In¬ 
structive  to  determine  whether  or  not  the  types  of  equipments  emphasized 
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were  sensitive  to  the  forms  of  the  algorithms  involved.  The  present 
researchers  offer  two  postulates:  (1)  "high-burner"  lists,  such  as 
IROS-LSC,  will  mostly  emphasize  jet  engines  and  avionics,  while  return¬ 
on-  investment  lists,  such  as  LIST-TD,  will  usually  emphasize  such  mun¬ 
dane  equipments  as  hydraulic  valves  and  actuators,  fuel  gauges,  etc.; 
(2)  the  types  of  equipments  emphasized  for  support  cost  reduction  will 
usually  be  significantly  different  that  those  emphasized  for  decreasing 
downtime  and  aborts. 

If  further  research  should  confirm  these  postulates,  then  the 
case  that  there  was  a  fundamental  and  significant  difference  in  the 
logic  behind  the  single  benefit  high-burner  and  the  combined  benefits 
return-on- investment  approaches  would  be  strengthened.  In  this  event 
the  objections  of  Meitzler  (11:1-2)  and  Woomer  (15:1)  to  the  logic  of 
the  high  burner  approach  should  be  given  serious  consideration.  Ideal¬ 
ly,  the  issue  would  best  be  resolved  by  the  consideration  of  the  actual 
Investments  and  benefits  made  In  past  reliability  improvements,  but  as 
Coleman  and  Edison  have  established  (5:17),  this  Is  at  best  a  difficult 
and  uncertain  undertaking. 

RECOMMENDATIONS  FOR  MANAGEMENT 

It  was  the  authors'  understanding  that  a  high  priority  Air  Force 
goal  was  to  make  sizable  reductions  in  the  support  costs  and  downtimes 
of  future  aircraft.  As  pointed  out  in  Chapter  1,  a  number  of  research¬ 
ers  suggested  that  concentrated  basic  research  to  improve  reliabilities 
of  carefully  selected  types  of  equipment  could  contribute  significantly 
towards  this  goal.  Several  of  the  researchers  felt  that  the  rational. 
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systematic  use  of  data  on  currently  fielded  aircraft  was  a  sound  ap¬ 
proach  to  selecting  the  types  of  equipments  for  research  concentration. 
As  was  shown  by  this  research,  however,  two  different,  systematic 


approaches  gave  significantly  different  results. 

Managers,  particularly  those  concerned  with  the  allocation  of 
resources  for  basic  research,  should  be  aware  that  for  at  least  one 
case,  the  types  of  equipment  one  would  emphasize  for  reliability  im¬ 
provement  are  significantly  different  when  one  uses  a  single- benefit, 
high-burner  algorithm  (IROS-LSD)  as  opposed  to  a  combined-benefit, 
return-on- investment  algorithm  (LIST-TD).  Managers  might  be  prudent  if 
they  considered  the  possibility  that  these  significant  differences  might 
not  be  unique  to  the  specific  algorithms  in  IROS-LSC  and  LIST-TD,  but 
were  the  result  of  fundamental  differences  in  thinking  and  approach. 
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APPENDIX  A 


T 


The  automated  IROS  system,  known  as  K051,  was  based  on  reported 
data  from  41  different  types  of  aircraft.  The  input  to  K051  was  ex¬ 
tracted  from  a  number  of  automated  data  systems  that  covered  not  only 
maintenance  activities  at  individual  air  bases,  but  repair  work  at  the 
central  depots,  etc.  KQ51  extracted  data  from  these  data  systems  quar¬ 
terly,  and  used  it  to  estimate  the  annual  logistics  support  costs  of 
some  150,000  authorized  work  unit  coded  equipments.  The  K051  process 
was  depicted  in  Figure  2. 

The  Logistic  Support  Cost  (LSC)  estimate,  while  primarily  based 
on  the  reported  data,  also  utilized  a  number  of  allocation  factors  and 
standards.  The  labor  cost,  for  instance,  was  the  product  of  the  re¬ 
ported  man-hours  and  a  standard  labor  rate  ($/hour).  Transportation 
costs  were  based  on  reported  weight  and  a  standard  cost  per  pound. 
Depot  repair  costs  of  each  equipment  type  were  based  on  the  product  of 
the  number  of  units  shipped  to  the  repair  depot  and  a  standard  cost  per 
repair  for  that  type  equipment. 

The  accuracy  of  the  IROS-LSC  estimate  and  the  data  on  which  it 
was  based  were  questioned  by  Fiorello  (6:22).  If  one  was  interested  in 
a  fairly  exact  estimate  of  the  annual  logistics  support  cost  of  a  spec¬ 
ific  work  unit  coded  equipment,  then  qualms  about  the  absolute  accuracy 
of  the  data  input  to  IROS  and  the  roughness  of  the  estimating  procedures 
would  be  understandable.  On  the  other  hand,  if  one  was  only  Interested 
in  what  types  of  equipments  (i.e.  an  ordinal  scaled  variable)  freq- 
quently  occur  in,  say  the  top  500  ranked  equipments,  then  the  accuracy 
required  for  this  objective  was  certainly  met  by  IROS-LSC. 
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FIGURE  2:  IROS-LSC  PROCESS 


As  shown  In  Figure  2,  the  top  ranked  IROS-LSC  equipments  can  be 
grouped  by  their  two  digit  work  unit  code  (2  digit  WUC).  There  were 
some  40  plus  unique  two  digit  WUCs  on  Air  Force  airplanes  and  heli¬ 
copters.  Finer  classification,  unfortunately,  was  not  consistent  from 
one  aircraft  to  the  next,  and  thus  the  two  digit  WUC  was  the  most  de¬ 
tailed  classification  available  by  work  unit  code.  As  mentioned  in  the 
introduction,  the  various  two  digit  WUCs  covering  engines  and  avionics 
were  the  equipment  types  with  the  greatest  frequency  of  occurrence  in 
the  IROS-LSC  500  top  ranked  equipments. 
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APPENDIX  B 
LIST-TD 
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The  LIST-TD  estimates  of  return-on-investment  were  based  on  31 
types  of  fielded  airplanes  (helicopters  were  excluded).  Data  reported 
on  the  equipment  on  these  aircraft  were  by  both  work  unit  code  (WUC)  and 
master  stock  number  (MSN).  An  entire  year  of  data  was  utilized  in  the 
LIST-TD  estimates.  Data  by  Master  Stock  Number  (MSN),  particularily 
stock-list  prices,  were  required  by  LIST-TD  to  estimate  the  investment 
required  to  Improve  reliability.  The  logistic  support  cost  (LSC)  por¬ 
tion  of  the  LIST-TD  was  exactly  identical  to  the  reported  IROS-LSC,  as 
the  latter  was  utilized  as  the  logistics  support  cost  estimate  in  LIST- 
TD.  The  LIST-TD  process  was  portrayed  in  Figure  3. 

The  fact  that  the  logistic  support  cost  (LSC)  estimates  were 
Identical  in  both  IROS-LSC  and  LIST-TD  was  a  fortuitous  coincidence  (due 
to  LIST-TD  being  a  Mdecendent“  of  IROS-LSC),  since  this  identity  was 
exploited  in  the  experimental  design. 

LIST-TD  estimates  of  downtime  and  abort  costs  were  based  on 
reported  downtimes,  aborts,  and  the  cost  of  the  entire  aircraft.  There 
was  no  general  agreement  as  to  how  downtimes  and  aborted  missions  should 
be  converted  Into  equivalent  costs,  or  even  whether  this  should  be  done 
at  all.  The  originators  of  LIST-TD  maintained  that  the  inclusion  of 
downtime  and  abort  costs  along  with  the  support  costs  was  essential  as 
these  costs  represent,  fully  one  half  of  the  overall  penalty  of  unreli¬ 
able  equipment  (7:3).  It  was  one  of  the  objectives  of  this  thesis  to 
determine  whether  or  not  the  Inclusion  of  downtime  and  abort  costs 
significantly  influence  the  types  of  equipment  suggested  for  reliability 
Improvement  research. 


The  Investment  in  LIST-TD  was  estimated  as  being  proportional  to 
the  cost  of  the  Installed  equipment. 

I  CX  UP^  x  N^r  x  QPA 

where 

I  was  the  estimated  investment 

UPg  was  the  unit  price  of  the  equipment  in  question  (stock  list 

price) 

Nac  was  the  number  of  aircraft  of  the  type  in  question 
QPA  was  the  quantity  (of  the  equipment)  per  aircraft 
The  LIST-TD  results  were  not  portrayed  as  being  absolute  es¬ 
timates  of  return-on-investment,  but  merely  an  approximate  rank  order 
list.  It  was  for  this  reason  that  a  proportionality  rather  than  an 
equality  was  given.  It  was  suggested  that  this  proportionality  should 
more  properly  vary  from  one  technological  area  to  the  next,  but  this 
refinement  was  not  pursed  by  the  developers  of  LIST-TD. 

The  LIST-TD  algorithm  was 

Rank  Order  ROI  by:  L-SC.  x  RL  ^  .(DTC  *  AC) 

where 

ROI  was  the  return-on-investment  rank  order 

LSC  was  the  IROS-LSC  estimate 

RL  was  the  estimated  aircraft  remaining  life 

DTC  was  the  downtime  cost  estimate 

I  was  the  reliability  Investment  estimate 

If  RL  was  set  to  any  nonzero  constant  value.  If  (DTC  +  AC)  was 
set  to  zero,  and  if  I  was  set  to  any  nonzero  constant  value,  then  with 
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Identical  data  Input,  the  top  ranked  IROS-LSC  and  LIST-TD  equipment 
types  will,  of  necessity,  be  Identical.  This  was  the  fortuitous  sit¬ 
uation  mentioned  earlier. 
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APPENDIX  C 
DETAILED  RESULTS 


The  results  for  the  primary  hypothesis  were  shown  in  Table  7. 
The  first  column  was  an  identifying  number.  The  second  column  was  the 
stock  class.  The  third  and  fourth  columns  were  the  observed  frequen¬ 
cies*  while  the  last  column  was  the  Chi-Square  value  for  individual 
stock  class.  Stock  Classes  that  were  significantly  different  at  the  .01 
level  of  significance  (i.e.  X  >  6.64)  were  annotated  with  an  as¬ 
terisk. 

For  (40-1)  X  (2-1)  »  39  degrees  of  freedom  and  a  .01  level  of 

2 

significance,  the  critical  X  value  was  62.4.  As  342.3  >  62.4,  the 
primary  null  hypothesis  was  rejected,  and  it  was  concluded  that  the 
difference  between  the  distribution  of  top  ranked  IROS-LSC  and  LIST-TD 
equipment  types  was  significant. 


NO.  SC  IROS-LSC  LI! _ 


1  6680 

2  6670 

3  6615 

4  6610 

5  6605 

6  6  340 

7  6220 

8  6210 
9  6140 

10  6115 

11  6110 

12  3895 

13  5865 

14  5841 

15  5826 

16  5821 

17  4920 

18  4820 

19  4810 

20  4320 


29  18.9* 

20  10.7* 

4  17.1* 

18  0.5 

21  15.2* 


NO.  SC 

21  2995 

22  2915 

23  2840 

24  2835 

25  2620 

26  1730 

27  1680 

28  1660 

29  1650 

30  1630 

31  1620 

32  1610 

33  1560 

34  1440 

35  1430 

36  1280 

37  1270 

38  1095 

39  1005 

40  0000 
TOTALS 


TABLE  7:  PRIMARY  HYPOTHESIS  RESULTS 


1  6.4 

6  6.0 

0  11.0* 

5  7.3* 

1  22.1* 

3  6.2 


IROS-LSC  LIST-TD  X* 


18 

10 

2.3 

19 

18 

0.0 

43 

10 

20.5* 

1 

4 

1.8 

9 

27 

9.0* 

1 

3 

1.0 

10 

61 

36.6* 

5 

30 

17.8* 

20 

52 

14.2* 

16 

28 

3.3 

2 

25 

19.6* 

7 

0 

7.0* 

42 

22 

6.2 

4 

0 

4.0 

17 

1 

14.2* 

18 

15 

0.3 

31 

6 

16.9* 

5 


The  "Miscellaneous"  combined  stock  class  (0000)  was  expanded  In 


Table  8  for 

SC 

reference . 

IROS-LSC 

LIST-TD 

SC 

IROS-LSC 

LIST-TD 

6685 

0 

2 

4440 

0 

1 

6645 

0 

1 

4310 

1 

0 

6250 

0 

1 

4140 

0 

1 

6105 

1 

1 

4130 

1 

2 

5995 

1 

0 

3040 

0 

1 

5990 

0 

1 

29  45 

0 

1 

5960 

1 

0 

2935 

1 

1 

5850 

0 

2 

2910 

0 

1 

5831 

1 

1 

1670 

0 

1 

.  5340 

0 

2 

1420 

0 

1 

4730 

0 

1 

1290 

0 

1 

TOTALS 

7 

25 

TABLE  8:  DETAILS  OF  MISC.  CLASS 


RETURN- ON- INVESTMENT 


The  analysis  for  the  return-on-investment  hypothesis  was  iden¬ 
tical  to  that  of  the  primary  hypothesis,  except  that  the  comparison  was 
between  IROS-LSC  and  LIST-LSC/I. 

The  results  were  shown  In  Tables  9  and  10.  The  number  of  stock 
classes  after  consolidation  was,  coincidently,  again  40,  so  that  same 
critical  table  value  applies  for  a  .01  significant  level.  As  the  test 
statistics  exceed  the  critical  value  (297.7  >62.4),  the  null  hypothesis 
was  rejected.  It  was  concluded  that  the  significant  difference  between 
top  ranked  IROS-LSC  and  LIST-TD  equipment  types  was  due  at  least  in  part 
to  the  return-on-investment  feature  of  LIST-TD. 
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NO. 

SC 

IROS-LSC 

LIST-TD 

r 

SC  IROS-LSC 

LIST-TD 

xf 

1 

6680 

4 

14 

5.5 

21 

2915 

19 

10 

2.8 

2 

6620 

4 

14 

5.5 

22 

2840 

43 

19 

9.3* 

3 

6615 

27 

8 

10.3* 

23 

2835 

1 

4 

1.8 

4 

6610 

14 

6 

3.2 

24 

2620 

9 

38 

17.9* 

5 

6605 

55 

18 

18.7* 

25 

1730 

1 

3 

1.0 

6 

6340 

0 

5 

5.9 

26 

1680 

10 

50 

26.7* 

7 

6220 

1 

17 

14.2* 

27 

1670 

0 

5 

5.0 

8 

6140 

5 

14 

4.3 

28 

1660 

5 

8 

0.7 

9 

6115 

9 

1 

6.4 

29 

1650 

20 

41 

7.2* 

10 

6110 

0 

4 

4.0 

30 

1630 

16 

46 

14.5* 

11 

5885 

11 

0 

11.0* 

31 

1620 

2 

26 

20.6* 

12 

5865 

18 

10 

2.3 

32 

1610 

7 

3 

1.6 

13 

5841 

25 

2 

19.6* 

33 

1560 

42 

24 

4.9 

14 

5826 

13 

11 

0.2 

34 

1440 

4 

0 

4.0 

15 

5821 

14 

3 

7.1* 

35 

1430 

17 

0 

17.0* 

16 

4920 

4 

2 

0.7 

36 

1280 

18 

21 

0.2 

17 

4820 

2 

3 

0.2 

37 

1270 

31 

13 

7.4* 

18 

4440 

0 

4 

4.0 

38 

1095 

6 

5 

0.1 

19 

4320 

12 

1 

9.3* 

39 

1005 

5 

10 

1.7 

20 

2995 

18 

5 

7.3* 

40 

0000 

_8 

32 

14.4* 

TOTALS 

500 

500 

297.7 

TABLE  9:  DETAILEO  RETURN-ON- INVESTMENT  RESULTS 
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sc 

7310 

6685 

6645 

6250 

6210 

6125 

6105 

5995 

5990 

5915 

5850 


IROS-LSC 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 


LIST-TD 

2 

1 

1 

1 

2 

1 

1 

1 

1 

1 

3 


SC 

5831 

5340 

4810 

4730 

4310 

4130 

3040 

2945 

2925 

2910 

1420 

TOTALS 


IROS-LSC 

1 

0 

1 

0 

1 

1 

0 

0 

0 

0 

0 

8 


LIST-TD 

1 

2 

1 

2 

2 

2 

2 

2 

1 

1 

1 

32 


TABLE  10:  DETAILS  OF  MISCELLANEOUS  CLASS 


TYPES  OF  BENEFITS 


The  final  hypothesis  was  analyzed  exactly  as  the  preceedlng  two 
except  the  comparison  was  between  LIST-LSC  and  LIST-AB,  the  added  bene¬ 
fit  model  (i.e.,  between  LSC/I  and  (DTC+AC)/I). 

The  results  were  shown  In  Tables  11  and  12.  The  null  hypothesis 

that  the  added  benefits  feature  of  LIST-TD  was  not  a  source  of  signifi- 

o 

cant  difference,  was  rejected  as  the  total  X  value  of  149.7  was  greater 
than  62.4.  The  alternative  hypothesis  that  added  benefits  were  a  signi¬ 
ficant  source  of  difference  was  accepted. 
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NO. 

SC 

IROS-LSC 

LIST-TD 

x£ 

NO. 

SC 

IROS-LSC 

LIST-TD 

l 

1 

6685 

1 

7 

4.5 

21 

4320 

1 

6 

3.6 

2 

6680 

14 

30 

5.8 

22 

4130 

2 

2 

0.0 

3 

6620 

14 

17 

0.3 

23 

2995 

51 

12 

2.9 

4 

6615 

8 

6 

0.3 

24 

2915 

10 

28 

8.5* 

5 

6610 

6 

21 

8.3* 

25 

2840 

19 

9 

3.6 

6 

6605 

18 

21 

0.2 

26 

2835 

4 

4 

0.0 

7 

6340 

5 

10 

1.7 

27 

2620 

38 

9 

17.9* 

8 

6220 

17 

5 

6.5 

28 

1730 

3 

4 

0.1 

9 

6210 

2 

3 

0.2 

29 

1680 

50 

58 

0.6 

10 

6140 

.  14 

1 

11.3* 

30 

1670 

5 

2 

1.3 

11 

6110 

4 

1 

1.8* 

31 

1660 

8 

32 

14.4* 

12 

5865 

10 

8 

0.2 

32 

1650 

41 

53 

1.5 

13 

5850 

3 

0 

3.0 

33 

1630 

46 

11 

21.5* 

14 

5841 

2 

6 

2.0 

34 

1620 

26 

20 

0.8 

15 

5826 

11 

4 

3.3 

35 

1560 

24 

27 

0.2 

16 

5821 

3 

3 

0.0 

36 

1280 

21 

18 

0.2 

17 

4920 

2 

12 

7.1* 

37 

1270 

13 

7 

1.8 

18 

4820 

3 

6 

1.0 

38 

1095 

5 

2 

1.3 

19 

4810 

1 

10 

7.4* 

39 

1005 

10 

4 

2.6 

20 

4440 

4 

2 

0.7 

40 

0000 

27 

19 

1.4 

TOTALS  500  500  149.7 


TABLE  11:  DETAILED  ADDED  BENEFITS  RESULTS 
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